A hard disk drive (HDD) stores data on one or more platters that have a plurality of concentric cylinders. The platters spin to move different locations under a read/write head(s). The data stored by the HDD may be organized in files, objects, records, tables, byte-streams, or in other ways. In the early days of computing, an HDD may have stored data for a single computer running a single process. When data was written in this scenario, there was only a single place from which the data would be provided and only a single HDD to which the data would be written. When data was going to be read, there was only one HDD from which the data could be read and only one place to which the data would be provided. Since there was only a single source and a single destination, attention could be paid to optimizing the data for write efficiency, for read efficiency, or for balancing write and read efficiency.
As computing progressed, HDDs began to store data for multiple processes associated with a computer. Decisions concerning where to store any particular data had to be made, either explicitly when the data was stored, or implicitly by configuring a program or utility. Decisions also had to be made concerning how to write data that arrived from different processes at different times. Similarly, decisions concerning to where data was to be provided had to be made. File systems and operating systems facilitated virtualizing the storage devices and improved some of the decision making.
As computing progressed even further, computers may have begun to interact on the storage side with networks of HDDs and on the processing side with networks of processes or computers. In this scenario, the locations available to store data expanded exponentially and the number of processes seeking to store or retrieve data also expanded exponentially. Still, decisions concerning where or how any particular piece of data was to be stored may have been made on an as-needed basis using heuristics that were designed in a single-disk/single-process era. This new problem is unique to data storage systems that interact with multiple data providers. It is not the type of problem that has existed for a long time or for which a paper and pencil solution exists since the writing and positioning of electronic data is involved.
For many applications, HDD performance may be a significant concern. HDD write performance may be affected by how data is written (e.g., randomly, sequentially) and HDD read performance may be affected by how data is read (e.g., randomly, sequentially). Traditionally, HDDs provide higher performance when accessed sequentially instead of randomly. However, the ability to perform random access is one of the criteria by which HDD are selected over other storage devices (e.g., tape drives) for a system. Conventional file systems, operating systems, applications, device drivers, or other processes that may access an HDD may be aware of the sequential versus random performance concern. However, interactions between ever increasing layers of virtualization with ever increasing complexity may frustrate attempts to organize data for sequential access.
Advances in HDDs have produced increases in capacity. However, the increases in capacity have not been matched with equal increases in HDD performance, even while storage system designers may equate capacity with performance. The disconnect between capacity and performance may frustrate systems designers. Additionally, the disconnect may lead to inefficiencies in data storage, which in turn may lead to over-spending and excessive energy consumption. The disconnect between capacity and storage caused by conventional storage heuristics may cause a conventional HDD to have excess capacity at a given performance level. In one extreme example, an HDD may be “short stroked”, where the fastest outer cylinders are used to store data to ensure the best possible transfer rate and the shortest seek latency. While the fastest outer cylinders may be over-utilized, middle and especially inner cylinders may be under-utilized, if even used at all.
Conventional systems may have sought to optimize write performance by using sequential writes, even for data from multiple processes. These conventional systems may have used log structured input/output (i/o) in attempts to produce more sequential writes. In log structured i/o, data is written to a device sequentially, in the order that the data arrives, regardless of a location intended for the data by the entity that provided the data to the log structured device. Thus, data from one source may be interleaved with data from other sources and the combined set of data written sequentially. Files may not be written contiguously. A log structured i/o device may maintain a block map that tracks where the data is actually written and where the entity that provided the data wanted the data to be written. Conventional log structured i/o tends to optimize writing over reading, which may lead to significant performance degradation when the data that was sequentially written needs to be read back in.
Log structuring may be implemented at the file system level. The file system may be responsible for assigning blocks to the data in the files managed by the file system. A file system may aggregate writes to files so that the writes are sequential and therefore faster than random writes. While the aggregation that leads to sequential writes may improve write speed, reads of a file that might be sequential in another file system may or may not be sequential in a log structured file system. For example, a file that might have been written contiguously in another system may not have been written contiguously in a log structured i/o device. Whether the read is sequential will depend on what else, if anything, was written at the same time as the file to be read. If the file to be read was the only thing written at the point in time when the file was written, then the file may have been written sequentially. However, if other data was written at the same time as the file to be read, then other data blocks may have been interspersed with the file to be read, which may in turn lead to non-sequential reads for the file. As HDDs store data for more and more processes associated with more and more computers, the likelihood that any single file will be written in isolation becomes less and less. Thus, HDD read performance may continue to suffer accordingly.
Log structuring has been employed in conventional solid state drives (SSD). Log structuring may be implemented at the block level in FLASH-based SSDs because this type of device is structured in pages that need to be erased before they can be rewritten. Log structuring is well suited to this type of device because writes are sequential and there is little to no penalty for non-sequential reads. When implemented in an SSD, log structuring may be referred to as a “FLASH translation layer.” The block translation map produced by log structuring may be stored on the SSD.
While log structuring may provide improved write performance for SSDs without the corresponding read performance degradation found in HDDs, the log structuring approach for SSDs may be compromised by SSD garbage collection. An issue may occur during garbage collection because live data in a page on an SSD may need to be moved before the page can be reclaimed. Thus, garbage collection for SSD may constrain long-term write throughput of FLASH-based SSD. This garbage collection issue may be associated with the observed phenomenon that while SSDs may be very fast early in their deployment, they experience gradual degradation in performance after deployment.